Learning with Maximum A-Posteriori Perturbation Models
نویسندگان
چکیده
Perturbation models are families of distributions induced from perturbations. They combine randomization of the parameters with maximization to draw unbiased samples. Unlike Gibbs’ distributions, a perturbation model defined on the basis of low order statistics still gives rise to high order dependencies. In this paper, we analyze, extend and seek to estimate such dependencies from data. In particular, we shift the modelling focus from the parameters of the Gibbs’ distribution used as a base model to the space of perturbations. We estimate dependent perturbations over the parameters using a hardEM approach, cast in the form of inverse convex programs. Each inverse program confines the randomization to the parameter polytope responsible for generating the observed answer. We illustrate the method on several computer vision problems.
منابع مشابه
An Efficient Method for Model Reduction in Diffuse Optical Tomography
We present an efficient method for the reduction of model equations in the linearized diffuse optical tomography (DOT) problem. We first implement the maximum a posteriori (MAP) estimator and Tikhonov regularization, which are based on applying preconditioners to linear perturbation equations. For model reduction, the precondition is split into two parts: the principal components are consid...
متن کاملAn Introduction to Inference and Learning in Bayesian Networks
Bayesian networks (BNs) are modern tools for modeling phenomena in dynamic and static systems and are used in different subjects such as disease diagnosis, weather forecasting, decision making and clustering. A BN is a graphical-probabilistic model which represents causal relations among random variables and consists of a directed acyclic graph and a set of conditional probabilities. Structure...
متن کاملOn Measure Concentration of Random Maximum A-Posteriori Perturbations
The maximum a-posteriori (MAP) perturbation framework has emerged as a useful approach for inference and learning in high dimensional complex models. By maximizing a randomly perturbed potential function, MAP perturbations generate unbiased samples from the Gibbs distribution. Unfortunately, the computational cost of generating so many high-dimensional random variables can be prohibitive. More ...
متن کاملMaximum a posteriori adaptation of network parameters in deep models
We present a Bayesian approach to adapting parameters of a well-trained context-dependent deep-neural-network hidden Markov models (CD-DNN-HMMs) to improve automatic speech recognition performance. Due to an abundance of DNN parameters but with only a limited amount of adaptation data, the posterior probabilities of unseen CD states (senones) are often pushed towards zero during adaptation, and...
متن کاملMaximum a Posteriori Tree Augmented Naive Bayes Classifiers
Bayesian classifiers such as Naive Bayes or Tree Augmented Naive Bayes (TAN) have shown excellent performance given their simplicity and heavy underlying independence assumptions. In this paper we prove that under suitable conditions it is possible to calculate efficiently the maximum a posterior TAN model. Furthermore, we prove that it is also possible to calculate a weighted set with the k ma...
متن کامل